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Summary 

If  we  are  to  present  instruction  that  is  available  anytime  and  anywhere,  takes  advantage  of  the  substantial 
tutorial  efficiencies  of  one  teacher  for  every  student,  and  is  affordable,  we  must  have  recourse  to  technology, 
specifically  computer  technology.  Such  technology  can  be  used  in  instructional  applications  that  range  from 
drill  and  practice  and  tutorial  dialogues,  to  multiplayer  simulations  and  games.  It  can  be  used  in  stand-alone 
modes  or  it  can  be  used  to  supplement  classroom  instruction.  It  can  be  used  by  individuals  or  by  groups.  In 
all  cases,  however,  it  must  take  account  of  the  current  state  of  the  learner,  the  eventual  state  of  the  learner 
that  the  instruction  is  intended  to  produce,  and  the  instructional  techniques  that  reliably  effect  transitions 
from  one  state  to  the  other.  Models  of  the  learner  that  represent  these  current  and  objective  states  must  to  an 
appreciable  extent  be  models  of  the  learner’s  cognition,  which  produces  the  skills,  performance,  and 
competence  needed  for  success  in  all  military  operations.  These  models  may  be  implicit  as  found  in 
intrinsically  programmed  instruction,  or  they  may  be  explicit.  Both  types  may  be  seen  in  technology-based 
instruction  from  its  beginning.  Early  explicit  models  were  largely  quantitative,  involving  fairly  simple 
instructional  paradigms,  but  fairly  elaborate  mathematics,  including  instructional  applications  of  optimal 
control  theory.  Current  efforts  are  more  concerned  with  qualitative  models,  1 9  of  which  are  briefly  described 
and  discussed.  These  models  all  contribute  to  some  degree  to  the  efficiency  and  effectiveness  of  technology- 
based  instruction.  However,  new  challenges  have  arisen  from  today’s  uncertain,  asymmetric  operational 
environment,  which  may  require  responses  that  cannot  be  foreseen  nor  well  prepared  for  in  advance.  Instead 
we  must  prepare  our  military  forces  and  personnel  to  expect  the  unexpected  and  be  prepared  for  it  with 
individual  and  collective  agility,  creativity,  and  adaptability.  These  qualities  are  fundamentally  cognitive  in 
nature  and  require  more  powerful  and  comprehensive  cognitive  models  if  they  are  to  successfully  serve  our 
programs  of  education,  training,  and  performance  aiding. 

Introduction 

This  paper  concerns  research  on  digital  representations  of  human  cognitive  processes  that  may  be  used  to 
develop  computer-mediated  learning  and  performance  aiding  systems.  We  refer  to  such  representations  as 
‘models’  of  human  cognition.  This  topic  turns  out  to  be  extensive  in  both  breadth  and  depth,  so  we  focused 
our  discussion  on  the  following  three  questions: 

What  is  the  military  value  of  these  models? 

What  is  their  current  state  of  development? 

What  is  their  relationship  to  instructional  systems  development? 

What  research  and  development  should  be  undertaken  to  advance  their  value  and  utility? 

Paper  presented  at  the  RTO  HFM  Symposium  on  “Advanced  Technologies  for  Military  Training", 
held  Genoa,  Italy!  3  -  15  October  2003,  and  published  in  RTO-MP-HFM-101 . 
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Many  valuable  contributions  are  being  made  by  researchers  who  are  modeling  neurological  activity  at  one 
end  of  the  behavioral  spectrum  and  by  researchers  who  are  modeling  human  physical  activity  and 
performance  at  the  other.  This  paper  is  aimed  somewhere  in  the  middle  of  these  efforts.  It  concerns  models, 
or  representations,  of  human  cognitive  processes  such  as  perception,  memory,  learning,  decision-making,  and 
problem  solving.  These  processes  arise  from  “micro”  neurological  activity  at  one  end  of  the  spectrum  and, 
in  turn,  produce  “macro”  physical  activity  and  performance  at  the  other.  Eventually,  research  and 
development  may  yield  models  that  unify  the  full  spectrum  of  human  behavior  from  neurons  to  psychomotor 
activity,  but  the  current  state  of  knowledge  limits  us  to  efforts  to  understand  and  model  components  of  this 
spectrum,  hence  our  focus  in  this  paper  on  one  such  component  -  cognition  -  which  seems  an  appropriate 
level  of  concern  for  learning  and  human  performance. 

Throughout  the  paper  we  refer  to  ‘the  user’.  This  is  intended  as  a  catchall  term  for  students,  decision  makers, 
technicians,  analysts,  and  anyone  else  who  may  be  using  computer  technology  for  education,  training,  and 
decision  and  performance  aiding.  The  paper  focuses  on  digital  representation  of  these  users’  cognitive 
processes. 

The  Military  Value  of  Cognitive  Modeling 

It  is  a  fact  both  obvious  and  frequently  neglected  that  human  competence,  which  is  a  product  of  human 
cognition,  is  essential  to  every  military  operation,  across  all  echelons  of  command  and  activity.  Its 
importance  is  perennially  evident  in  the  conduct  of  military  operations.  Even  in  the  increasingly  technology- 
saturated  environments  of  modem  operations,  human  competence  is  needed  to  launch  and  control  systems  in 
space,  operate  and  maintain  robotic  vehicles,  deploy  remote  sensors  and  systems  in  contested  territory,  and 
so  forth.  In  short,  there  are  no  unmanned  systems.  Without  competent  people  to  operate,  maintain,  and 
deploy  our  materiel  assets,  investments  in  them  will  return  little  and  may  in  effect  be  wasted.  Given  the  wide 
availability  of  technology  and  the  ease  with  which  it  can  be  obtained,  human  competence  may  increasingly 
account  for  the  difference  between  success  and  failure  in  military  operations.  Its  availability  to  commanders 
anytime  and  anywhere  it  is  needed  is  a  matter  of  first  importance. 

How  might  we  ensure  this  availability?  Training  (and  its  performance  aiding  analog)  provides  one  means  to 
accomplish  this  objective,  particularly  if  it  can  be  delivered  anytime  and  anywhere.  For  example,  we  might 
supply  each  user  with  an  omnipresent  tutor.  Such  tutoring  is  probably  best  done  by  a  human  who  possesses 
expertise  in  the  relevant  subject  matter,  a  comprehensive  range  of  tutorial  techniques,  and  sufficient 
knowledge  of  the  user  to  identify,  establish,  and  sustain  in  that  individual  the  precise  human  competence 
needed.  Research  has  shown  such  tutoring  to  be  extremely  effective,  producing  an  often-noted  two  standard 
deviations  of  improvement  over  less  accessible  and  less  effective  classroom  instruction  (Bloom,  1984).  Such 
an  approach  has  been  called  an  instructional  imperative  and  an  economic  impossibility.  It  would  be 
maximally  effective  but  remains  unaffordable  because,  for  many  obvious  reasons,  we  cannot  supply  every 
user  with  a  human  tutor.  This  situation  creates  a  gap  between  what  is  needed  and  what  we  can  afford.  As  in 
many  other  endeavors,  we  are  attempting  to  apply  technology  to  fill  this  gap. 

The  research  evidence  suggests  that  such  applications  of  technology  can  succeed.  In  nearly  300  studies 
comparing  classroom  (one  teacher,  many  students)  with  computer-mediated,  individualized  instruction  (one 
computer  teacher,  one  student)  across  many  different  settings  and  subject  matters,  we  find  a  ‘rule  of  thirds’ 
emerging.  That  is  to  say  that,  compared  to  classroom  instruction,  technology-based  instruction  costs  about  a 
third  less  and,  additionally,  either  increases  achievement  by  a  third  when  instructional  time  is  held  constant, 
or  decreases  time  to  reach  constant  levels  of  achievement  by  about  a  third.  More  detailed  discussions  of 
these  data  have  been  presented  by  Fletcher  (1997,  2002),  Foster  and  Fletcher  (2002),  Kulik  (1994),  Niemiec, 
Sikorski,  &  Walberg  (1989),  and  others.  The  primary  payoff  for  military  operations  is,  of  course,  the  more 
rapid  and  reliable  preparation  of  personnel  to  perform  operational  duties,  producing  significant  payoffs  for 
resource  expenditures,  readiness,  and,  most  importantly,  operational  effectiveness. 
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Similar  research  evidence  exists  in  support  of  technology  used  to  aid  performance  and  decision-making.  For 
instance,  technicians  with  only  general  training  have  been  found  to  perform  as  well  as  specialists  (who 
required  time-consuming  and  expensive  training)  if  they  are  provided  hand-held  or  wearable  performance 
aids  (e.g.,  Fletcher  and  Johnston,  2002;  Joyce,  2001;  Wisher  and  Kinkaid,  1989).  These  aids  contribute  to 
military  readiness  and  effectiveness  not  only  by  releasing  individuals  earlier  for  operational  duty,  but  also,  by 
enhancing  human  competence  for  maintaining,  operating,  and  deploying  materiel  assets  —  thereby 
significantly  enhancing  materiel  readiness. 

Costs  also  matter,  of  course.  For  instance,  the  United  States  military  spends  about  $4  billion  a  year  on 
specialized  skill  training.  This  is  the  training  provided  after  “basic”  or  accession  training  to  qualify 
personnel  for  the  many  technical  jobs  (e.g.,  wheeled  vehicle  mechanics,  radar  operators,  avionics  technicians, 
oceanographers,  medical  technicians)  needed  to  perform  military  operations.  It  does  not  include  aircraft  pilot 
training,  field  training,  or  factory  training,  which  are  covered  in  separate  cost  categories.  If  the  US  were  to 
reduce  by  30  percent  the  time  to  train  20  percent  of  the  personnel  undergoing  specialized  skill  training,  it 
would  save  over  $250  million  per  year.  If  it  were  to  do  so  for  60  percent  of  the  personnel  undergoing 
specialized  skill  training,  it  would  save  over  $700  million  per  year  (Foster  and  Fletcher,  2002).  These  are 
appreciable  savings  by  most  standards. 

What  do  these  analyses  and  observations  have  to  do  with  cognitive  modeling?  Effective  education,  and 
training  must  start  with  a  dynamically  updateable  understanding,  or  model,  of  the  current  state  of  the  user,  a 
model  of  the  knowledge,  skills,  and  abilities  the  user  should  attain,  and  instructional  techniques,  strategies, 
and  processes  for  getting  from  one  to  the  other.  This  sort  of  modeling  occurs  in  classroom  learning  where 
teachers  continually  assess  what  their  students  know,  the  level  or  degree  to  which  they  know  it,  and  the  most 
efficient  ways  to  progress  in  achieving  instructional  goals.  As  discussed  in  the  next  section,  this  modeling  is 
also  found,  both  implicitly  and  explicitly  in  effective  technology-based  instruction. 

Similar  modeling  processes  are  also  required  to  support  performance  and  decision  aiding,  even  though  the 
emphasis  in  these  applications  is  on  problem  solution  rather  than  learning.  A  model  of  the  user  is  needed  to 
provide  advice  that  can  be  understood  and  carried  out,  as  well  as  a  model  of  the  system  or  situation  with 
which  the  user  is  interacting,  and  the  ability  to  maintain  something  similar  to  an  instructional  dialogue  to 
help  the  user  identify  correct  solutions  or  decisions. 

In  both  cases,  concern  with  the  knowledge,  skills,  and  abilities  that  comprise  human  competence  leads  us  to 
human  cognition  and  the  need  to  map  current  cognitive  states  onto  goal  cognitive  states  and  determine  what 
must  be  done  next.  This  presents  severe  difficulties  for  classroom  instruction  (one  teacher,  many  students). 
For  instance,  a  problem  arises  from  the  degree  to  which  students  in  a  typical  classroom  vary  in  their  prior 
knowledge,  abilities,  and  learning  progress.  Research  suggests  variation  by  about  a  factor  of  five  (e.g., 
Como  &  Snow,  1986;  Gettinger  &  White,  1980;  Gustaffson  &  Undheim,  1996;  Tobias,  1989).  Especially 
notable  for  military  applications  is  the  observation  that  variability  in  prior  knowledge  increases  with  age  and 
may  be  more  important  in  determining  progress  in  such  post-secondary  venues  as  military  training  than  it  is 
for  students  in  their  earlier  years  of  schooling.  Such  variability  suggests  the  importance,  for  both  efficiency 
and  effectiveness  in  military  education,  training,  and  performance  aiding,  of  tailoring  them  to  the  specific 
needs  of  individual  users. 

Assessment  of  cognition  in  classroom  instruction  is  necessarily  both  informal  and  imprecise.  If  we  seek  to 
achieve  human  performance  outcomes  reliably,  anytime,  anywhere,  and  affordably,  we  must  have  recourse  to 
technology.  If  we  are  to  use  computer  technology  to  achieve  these  ends  we  must  be  able  to  represent  —  or 
model  in  digital  form  —  current  cognitive  states,  goal  states,  and  ongoing  progress  from  one  to  the  other.  The 
empirical  results  discussed  above,  arising  from  technology-based  education,  training,  and  performance 
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aiding,  suggest  that  to  some  degree  we  have  been  successful  in  doing  this.  The  question  then  naturally  arises 
as  to  how  well  our  ability  to  implement  and  use  such  models  meets  the  need  for  them. 

The  Current  State  of  Cognitive  Modeling 

Implicit  Cognitive  Models 

Cognitive  models  are  implemented  both  implicitly  and  explicitly  in  technology -based  instruction.  Consider 
the  following  sample  instructional  item,  which  is  typical  of  much,  perhaps  most,  computer-mediated 
instruction: 

In  the  multiplication  3x4=12,  the  number  12  is  called  a _ . 


A. 

Factor 

{Branch  to  remedial  XI } 

B. 

Quotient 

{Branch  to  remedial  X2} 

C. 

Product 

{Reinforce,  go  to  next} 

D. 

Power 

{Branch  to  remedial  X3} 

In  this  item,  the  system,  the  computer  instructor,  assumes  that  a  student  responding  “A”  misunderstands  the 
meaning  of  ‘Factor,’  and  lacks  an  understanding  of  ‘Product’,  or  both.  The  student  will  be  branched  to  some 
instructional  materials  intended  to  correct  one  or  the  other  of  these  cognitive  states  and  then  returned  to  this 
item  or  one  similar  to  it.  The  same  type  of  remedial  approach  is  applied  to  responses  of  “B”  and  “D”.  A 
student  responding  with  “C”  may  be  rewarded,  ‘reinforced’,  with  encouraging,  or  positive,  feedback  and  is 
sent  on  to  whatever  item  will  continue  progress  toward  the  instructional  goal(s),  an  action  which  may  by 
itself  constitute  positive  reinforcement. 

The  above  item  appears  in  an  article  by  Norman  Crowder  written  for  Automated  Teaching,  a  book  that  was 
published  in  1959.  We  may  assume  the  use  of  cognitive  models  is  a  recent  innovation  in  technology -based 
instruction,  but  there  is  a  model  of  cognition  and  instructional  progress  evident  in  this  approach.  It  covers 
transitions  from  unlearned  to  learned  states  and  illustrates  what  Crowder  called  ‘intrinsic’  programming. 
This  approach  stands  in  contrast  to  the  expensive  and  difficult-to-prepare  ‘extrinsic’  programming  advocated 
by  B.  F.  Skinner  (e.g.,  1954),  and  for  good  reasons  of  economy  and  utility  it  is  the  dominant  approach  (still) 
in  use  today  in  technology-based  instruction.  It  may  be  found  covering  many  subject  matters,  posing 
questions  following  text  paragraphs,  graphic  displays,  simulations,  audio  presentations,  video  sequences, 
and/or  other  sources  of  instructional  content,  but  the  underlying  logic  remains  the  same  as  Crowder’s  original 
-  display  something,  elicit  a  response,  and  branch  to  remedial  or  reinforcing  material  depending  on  the 
response. 

In  order  to  prepare  an  item  such  as  the  above,  a  developer  must  both  anticipate  and  prepare  responses  for 
several  discrete  cognitive  states,  represented  by  the  correct  answer  (response  C)  and  the  ‘distractors’ 
(responses  A,  B,  and  D)  to  the  item.  The  cognitive  model  represented  by  these  states  is  static,  implicit,  and 
limited,  but  it  is  there.  The  main  difference  between  the  cognitive  modeling  in  Crowder’s  (and  Skinner’s) 
approaches  and  the  cognitive  modeling  being  developed  today  is  that  the  earlier  models  for  intrinsic  (and 
extrinsic)  programming  were  implicit,  embodied  in  the  program  of  instruction,  whereas  today  we  are 
attempting  to  use  more  explicit  models  of  cognition  that  we  can  abstract,  express,  and  validate  separately 
from  the  systems  in  which  they  are  used. 

Explicit  Cognitive  Models:  Quantitative 

These  more  explicit  cognitive  models  are  being  used  for  such  applications  as  intelligent  tutoring  systems  and 
the  human  behavior  modeling  we  need  to  generate  computer  (automated)  military  forces  for  constructive  and 
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ORGANIZATION 


virtual  simulation.  Explicit  models  of  cognition  were  also  applied  early-on,  in  the  1960s.  They  were  simple 
and  intended  to  account  for  rudimentary  learning  objectives  that  could  be  reduced  to  something  like  the 
substantial  amounts  of  stimulus-response,  associative  pairing  required  to  learn  such  material  as  arithmetic 
‘facts’  (addition,  subtraction,  multiplication  tables),  second  language  vocabulary,  and  technical  jargon 
(names  and  functions  of  biological  or  mechanical  structures).  Nonetheless,  they  led  to  sophisticated  and 
effective  instructional  approaches,  and  the  line  of  research  needed  to  determine  the  full  range  of  learning 
situations  and  objectives  to  which  they  could  be  applied  was  begun  but  left  unfinished  and  is  rarely  found 
today. 

As  an  example  of  this  approach  (and  its  use  of  cognitive  models)  consider  the  following  model  of  learning 
(adapted  from  Paulson,  1973)  which  attempted  to  account  for  the  probability  that  a  particular  item  for  a 
particular  learner  would  transition  from  the  unlearned  state  (U),  to  either  a  short-term  learned  state  (S),  i.e., 
present  in  working  memory,  or  to  a  long-term  learned  state  (L),  i.e.,  stored  in  long-term  memory: 


State  on  Trial  n+1 

State  on 
Trial  n 

L 

S 

U 

P(correct) 

L 

1 

0 

0 

1 

S 

c 

1-c 

0 

1 

u 

a 

b 

1-a-b 

S 

In  words: 


•  If  a  learned  item  (state  L)  is  presented,  then: 

-  With  probability  =  i,  it  stays  there. 

•  If  an  unlearned  item  (state  U)  is  presented,  then: 

-  With  probability  =  a,  it  will  transition  to  long  term-memory  and  the  learned  state, 

-  With  probability  =  b,  it  will  transition  to  a  short-term  state  (S)  in  working  memory  from  which  it  can  either 
be  learned  or  forgotten,  and  finally, 

-  With  probability  =  1-a-b,  it  will  remain  unlearned. 

•  If  an  item  is  in  short-term,  working  memory  (state  S),  then: 

-  With  probability  =  c,  it  will  transition  to  long-term  memory  and  the  learned  state,  otherwise, 

-  With  probability  =  1-c,  it  will  remain  in  the  short-term  state. 

•  An  item  in  the  short-term  state  will  not  slip  back  to  the  unlearned  state. 

This  formulation,  which  is  based  on  Paulson’s  (1973)  discussion,  accounts  for  guessing.  As  shown  in  the 
right-most  column  above,  he  assumed  a  probability  =  g  (presumably  for  ‘guessing’)  of  a  correct  answer  to  an 
unlearned  item,  but  a  probability  =  1_  for  a  correct  answer  to  an  item  in  the  learned  or  short-term  state.  The 
parameters  are  estimated  for  each  item-student  combination. 

A  key  feature  of  this  model  is  that  it  accounts  for  items  that  are  not  presented  on  a  trial.  In  Paulson's 
formulation  —  based  on  Rumelhart’s  General  Forgetting  Theory  (1967)  —  when  an  item  is  not  presented, 
transitions  between  states  are  expected  to  occur  in  accord  with  the  following  transition  matrix: 
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State  on  Trial  n+1 

State  on 
Trial  n 

L 

s 

u 

L 

1 

0 

0 

S 

0 

1-f 

f 

u 

0 

0 

1 

In  words,  when  an  item  is  not  presented: 

•  If  it  is  in  the  learned  or  unlearned  state,  it  stays  there; 

•  If  it  is  in  the  short-term  state,  it  may  regress  to  the  unlearned  state  with  probability  f  or  remain  in  the  short 
term  state  with  probability  1-f. 

Formulations  such  as  this,  which  are  based  on  explicit  transition  models  of  memory,  led  to  an  instructional 
strategy  that  has  been  proven  optimal  in  maximizing  the  number  of  items  learned  in  the  total  time  set  aside 
for  instruction,  T,  and  allowing  for  a  predetermined  number  of  items,  N,  to  be  presented  in  a  single  session 
(e.g.,  Atkinson  and  Paulson,  1972).  The  optimal  solution  determines  which  N  items  to  present  to  a  particular 
student  so  that  the  total  number  of  items  the  student  learns  is  maximized  at  time  T.  The  solution  is  roughly 
the  following: 

1 .  Before  each  trial,  identify  the  item  or  items  in  N  that  have  received  the  fewest  number  of  correct 
responses  since  the  last  error. 

2.  If  only  one  item  is  identified,  present  that  item. 

3 .  If  more  than  one  item  is  identified,  select  from  this  group  the  item  or  items  that  have  been  presented 
the  fewest  number  of  times. 

4.  If  only  one  item  remains,  present  that  item. 

5.  If  more  than  one  item  remains,  select  one  at  random  and  present  it. 

This  description  does  not  describe  how  items  that  have  reached  criterion  in  the  current  pool  of  N  items  can  be 
optimally  replaced  with  new  items.  Such  procedures  have  been  discussed  by  Atkinson  and  Paulson  (1972) 
and  Chant  and  Atkinson  (1973). 

Quantitative  models  of  this  sort  continue  to  be  used  in  technology-based  as  particularly  evidenced  by  efforts 
to  apply  Bayesian  networking  to  assess  the  cognitive  states  of  learners  (e.g.,  Van  Lehn  &  Niu,  2001).  These 
models  use  Bayes’  theorem  to  work  backward  from  users’  responses  to  determine  the  probabilities  that  they 
are  using  (perhaps  have  learned)  specific  cognitive  processes.  This  approach  can  lead  to  quite  sophisticated 
models  of  learners’  knowledge  and  skills. 

Three  points  may  be  worth  making  here:  (1)  Both  implicit  and  explicit  models  of  cognition  and  cognitive 
processes  have  been  used  in  technology-based  instruction  from  its  beginning;  (2)  Fairly  simple  cognitive 
models  for  fairly  simple  instructional  paradigms  can  lead  to  sophisticated  and  effective  instructional 
strategies;  and  (3)  This  approach  remains  a  promising  line  of  quantitative  research  that  deserves  to  be 
explored  more  fully. 

Explicit  Cognitive  Models:  Qualitative 

A  line  of  research  and  development  in  cognitive  modeling  that  has  been  more  vigorously  pursued  in  recent 
years  is  less  quantitative  than  the  above  models,  but  the  range  of  cognition  covered  tends  to  be  more 
comprehensive  and  can  thereby  be  used  to  meet  a  wider  range  of  learning  objectives.  This  work  typically 
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comes  under  the  heading  of  ‘human  behavior  modeling’  and  is  increasingly  used  in  the  development  of 
simulations  for  training  personnel  and  units,  analyzing  tactical,  operational,  and  strategic  alternatives,  and 
designing,  developing,  and  acquiring  military  materiel. 

We  are  fortunate  that  a  number  of  systematic  and  comprehensive  analyses  of  these  models  have  recently 
appeared  such  as  those  by  Pew  and  Mavor  (1998),  who  reviewed  11  such  models,  Ritter,  et  al.  (2002),  who 
reviewed  7  models  not  covered  by  Pew  and  Mavor,  and  Morrison  (2003),  who  reviewed  19  such  models. 

The  models  selected  for  analysis  in  these  reviews  were  intentionally  devised  to  be  implemented  in  digital 
form  -  in  computer  algorithms.  Doing  this  for  any  model  is  a  significant  demonstration.  If  a  model  can  be 
represented  in  an  algorithm,  it  can  be  tested.  Using  its  algorithmic  representation  to  capture  and  test 
cognitive  processes  can  significantly  enhance  both  our  knowledge  of  these  processes  and  the  effectiveness  of 
our  education,  training,  and  performance  aiding  applications.  Diagnostic  information  indicating  where  the 
model  is  correct,  will  demonstrate  the  validity  of  the  model,  and  indicating  where  it  is  incorrect,  will  suggest 
where  the  model  must  be  modified  to  account  for  the  full  range  of  human  cognition.  Significant  scientific 
and  technological  advances  can  arise  from  information  of  this  sort,  as  well  as  substantial  improvements  in 
our  ability  to  educate,  train,  and  assist  military  personnel. 

As  Morrison  (2003)  points  out,  most  of  these  models  are  systems  of  if-then,  condition-response 
(‘production’)  rules  that  simulate  cognitive  structures  and  processes.  The  19  models  he  reviewed,  which 
provide  a  snapshot  of  the  current  state  of  human  cognition  and  behavior  representation,  are  summarized  in 
Table  1. 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects  in  Models  Reviewed  by  Morrison  (2003) 


Model  Name 

Summary  Description 

Reference(s) 

Atomic  Components  of 
Thought  (ACT) 

Intended  to  provide  a  unified  theory  of  mind  and 
a  design  basis  for  instructional  environments 
(e.g.,  intelligent  tutors,  computer  generated 
forces)  and  human  interfaces.  Distinguishes 
between  declarative  knowledge  (represented  with 
semantic  networks)  and  procedural  knowledge 
(represented  using  if-then  rules). 

(Lebriere,  2002) 
(Anderson, 
Bothell,  Byrne, 
and  Lebriere, 
2002) 

Adaptive  Resonance 
Theory  (ART) 

Family  of  neural  net  models  designed  to  explain 
sensory -cognitive  processes  (e.g.,  perception, 
recognition,  attention,  reinforcement,  recall,  and 
working  memory).  Postulates  bottom  up  (e.g., 
perceptions)  and  top  down  (e.g.,  expectations, 
attention  control)  functions  in  working  memory 
that  interact  to  produce  learning. 

(Grossberg, 

1976a;  1976b) 
(Krafft,  2002) 
(http://web.umr. 
edu/~tauritzd/art 
) 

Architecture  for 

Procedure  Execution 
(APEX) 

Intended  to  reduce  time  and  effort  needed  to 
develop  models  of  human  performance  in 
complex,  dynamic  environments  such  as 
simulations,  explorations  of  human  performance 
theories,  and  assessments  of  equipment  design  on 
human  performance.  Includes  goal-directed 
action  selection  for  tasks  and  procedures  and 
resource  allocation  for  perceptual  (mostly 
visual),  cognitive,  and  psychomotor  functions. 

(Freed, 

Dahlman,  Dalai, 
and  Harris, 

2002) 

(http:// 

www.andrew.c 

mu.edu/~bj07/ap 

ex) 

Business  Redesign 
Agent-Based  Holistic 
Modeling  System 
(Brahms) 

Models  social  as  well  as  man-machine 
interactions.  Uses  agents  to  model  interactions 
among  physically  dispersed  groups  (e.g.,  teams), 
and  if-then  rules  (“detectables”  and  “beliefs”)  to 
model  decision  making  (via  “thoughtframes”) 
and  behavior  within  the  groups.  Emphasizes 
ethnographic  analyses  and  socio-technical  work 
practices,  activities  shaped  by  socio-technical 
environment,  and  constructivist,  situated 
cognition  to  model  cognition  and  behavior. 

(Sierhuis  and 
Clancey,  1997) 
(Clancey,  Sachs, 
Sierhuis,  and 
van  Hoof,  1998) 
(Acquisti, 

Clancey,  van 
Hoof,  Scott,  and 
Sierhuis,  2001) 

Cognition  and  Affect 
Project  (CogAff) 

{with  associated 
SimAgent  toolkit} 

Conceptual  space  for  describing  cognitive 
architectures.  Integrates  emotional  with 
cognitive  processes.  Incorporates  three  layers  of 
cognition  (reactive,  deliberative,  and  reflective  or 
meta-co gnitive),  three  layers  of  information 
processing  (perception,  central  processing,  and 
action),  and  three  types  of  emotions  (primary 
based  on  reaction,  secondary  based  on 
deliberation,  and  tertiary  based  on  reflection)  all 
producing  different  perceptual,  memory,  and 
motor  functions. 

(Sloman,  2001; 
2003) 

(http://www.cs.b 

ham.ac.uk/~axs/ 

cogaff.html) 
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Model  Name 

Summary  Description 

Reference(s) 

Cognition  as  a  Network 
Of  Tasks  (COGNET) 
{with  associated  GINA 
and  iGEN™  toolkits) 

Intended  for  cognitive  task  analysis  and 
description  of  work  domains  in  multi-task 
environments  requiring  contemplative,  decision- 
oriented,  open-ended  responses.  Uses  three 
subsystems  to  represent  information  processing 
(sensory/perceptual,  mental  modeling, 
action/motor),  four  forms  of  if-then  rule-based 
task  knowledge  (goal  directed  task  hierarchies, 
perceptual  demons  to  guide  attention,  blackboard 
for  organizing  declarative  information,  and 
possible  actions  linked  to  time  and  resource 
requirements),  and  meta-cognitive  functions. 
Allows  interfacing  with  other  applications. 

(Zachary, 

Campbell, 

Laughery, 

Glenn,  and 
Cannon-Bowers, 
2001) 

(http://www.chii 

nc.com/cogneth 

ome.shtml) 

Cognitive  Complexity 
Theory  (CCT) 

{with  associated 

GLEAN 3  toolkit} 

Focused  on  human  interface  design,  human- 
computer  interaction,  and  sequential  task 
performance.  Employs  device  models  (transition 
networks),  user  models  (sequentially  executed  if- 
then  rules,  the  fundamental  CCT  units  of 
cognition,  retrieve  from  long-term  memory),  and 
mental  operators  to  represent  covert  cognitive 
processes.  Long-term  memory 

(Kieras  and 
Poison,  1985) 
(Kieras,  1999) 

Cognitive  Objects  within 
a  Graphical 
EnviroNmentT 
(COGENT) 

Intended  solely  to  provide  tools  (via  a  visual 
programming  environment  that  evolves  with  the 
model  being  built)  for  cognitive  modeling, 
assuming  functional  modularity  (cognition  as 
interaction  among  semi-autonomous  subsystems) 
and  using  low-level  processing  components. 

(Cooper,  Yule, 
and  Sutton, 

1998) 

(Y ule  and 

Cooper,  2000) 

(http://cogent.ps 

yc.bbk.ac.uk) 

Concurrent  Activation- 
Based  Production 

System  (CAPS) 

Hybrid  model  for  central  cognitive  functions 
(e.g.,  reading  comprehension).  Primary  focus  is 
on  modeling  patterns  of  brain  activation  patterns 
in  high-level  cognition  via  if-then  rules  for 
specific  areas  of  the  brain  and  associative 
networks  for  cognitive  subsystems.  Total 
activation  in  working  memory  is  capped, 
concerned  exclusively  with  declarative 
knowledge  (facts),  but  with  different  limits  for 
different  individuals.  Long-term  memory 
includes  procedural  and  declarative  knowledge. 

(Just,  Carpenter, 
and  Varma, 

1999) 

(http://coglab.ps 
y.cmu.edu/proje 
cts  set.html) 

Construction-Integration 

Theory 

(C-l  Theory) 

Uses  a  symbolic  theory  of  sentence 
comprehension  and  propositions  (actions  and 
objects  of  the  action)  stressing  goal  formation  to 
provide  a  general  model  of  cognition. 
Comprehension  progresses  from  approximations 
to  verified  integration  through  mutually 
reinforced  associations  and  spreading  activation 
in  memory.  Extended  to  cover  comprehension  of 
novel  computer  interfaces  (Linked  model)  and 
new  websites  (CoLiDeS  model)  and  to 
incorporate  concepts  from  Latent  Semantic 

(Kintsch,  1998) 
(Landauer  & 
Dumais,  1997) 
(Kitaj  ima  & 
Poison,  1997) 
(Kitaj  ima, 
Blackmon,  & 
Poison,  2000) 
(http://psych- 
www.colorado.e 
du/ics) 
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Model  Name 

Summary  Description 

Reference(s) 

Analysis  (LSA)  used  to  derive  meaning  from 
text. 

Distributed  Cognition 
(DCOG) 

Intended  to  model  individuals’  expert  behavior 
with  agents  that  use  multiple  strategies  to 
respond  to  a  complex  environment  (air-traffic 
control).  Based  on  a  two  dimensional  space: 
Abstraction  with  three  levels  (skill-based 
responses  to  signals,  rule-based  responses  to 
signs,  and  knowledge-based  responses  to 
symbols)  and  Decomposition  (ranging  from 
individual  component  to  total  system 
processing).  Processing  within  this  space 
depends  on  level  of  expertise,  workload 
environment,  and  an  individual’s  preferred  level 
of  engagement. 

(Eggleston, 
Young,  & 
McCreight, 

2000) 

(Eggleston, 
Young,  and 
McCreight, 

2001) 

Executive 
Process/Interactive 
Control  (EPIC) 

Intended  to  model  details  of  peripheral  cognitive 
processes,  input  (perception)  and  output 
(psychomotor  responses)  to  inform  human- 
system  interface  design  by  predicting  the  order 
and  timing  of  responses.  Includes  long-term 
storage  of  declarative  and  procedural  knowledge 
and  working  memory  for  assessing  their 
application.  Capacity  and  retrieval  limitations 
arise  only  from  perceptual  and/or  psychomotor 
systems,  not  from  central  memory  store. 

(Kieras  & 

Meyer,  1995) 
(http://www.eec 
s.umich.edu/~ki 
eras/epic.html) 

Human  Operator 
Simulator  (HOS) 

Intended  to  inform  human-system  interface 
design  by  modeling  human  performance  based 
on  the  sequence  and  timing  of  subtasks  organized 
in  networks.  Uses  simulation  objects 
(configuration  of  displays  and  controls),  task 
networks  (if-then  rules  selecting  verb-object 
pairs  used  to  manipulate  the  objects),  and  micro¬ 
models  (times  to  complete  required  subtasks 
involving  perception,  information  processing, 
and  psychomotor  responses)  to  determine  human 
response  times. 

(Wherry,  1976) 
(Harris, 
lavecchia,  & 

Dick,  1989) 
(Glenn, 

Schwartz,  & 

Ross,  1992) 

Man-machine  Integrated 
Design  and  Analysis 
System  (MIDAS) 

Intended  to  inform  human-system  interface 
design  by  modeling  individuals  and  interactions 
among  individuals  in  performing  multiple, 
concurrent  tasks.  Uses  sensory  input  (operators 
and  perceivable  -  detectable,  recognizable,  and 
identifiable  -  objects),  memory  (with  declarative 
-  beliefs  in  long-term  memory,  contexts  in 
working  memory  -  and  procedural  components), 
decision-making,  attention  (with  limitations  on 
processing  resources),  situation  awareness 
(actual  and  perceived),  and  psychomotor  output 
to  model  human  operator  limitations  and 
capabilities. 

(Corker  & 

Smith,  1993) 
(Hart,  Dahn, 
Atencio,  & 

Dalai,  2001) 
(http://caffeine.a 
rc.nasa.gov/mid 
as/index.html) 
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Model  Name 

Summary  Description 

Reference(s) 

Micro  Systems  Analysis 
Of  Integrated  Network 

Of  Tasks  (Micro  Saint) 
{May  include  the 
Integrated  Performance 
Modeling  Environment 
(1PME),  using  HOS 
micro-models,  and 
WinCrew  for  estimating 
workload} 

Simulation  tool  that  uses  a  detailed  task  analysis 
to  decompose  human  performance  into  a 
networked  hierarchy  (with  branching  logic  and 
sequential  dependencies)  of  discrete  tasks  and 
subtasks  for  which  performance  estimates  can  be 
validated.  Network  consists  of  subtask  nodes 
(with  launching  conditions,  time  to  complete, 
and  effects)  and  relationships  (that  may  be 
probabilistic,  tactical  requiring  a  threshold  value, 
or  multiple  initiating  more  than  one  subtask). 
Designed  to  communicate  with  other  models  and 
applications  through  middleware. 

(Laughery  & 
Corker,  1997) 

Operator  Model 
Architecture  (OMAR) 
{Uses  Developers 
Interface,  a  graphics 
toolkit,  for  developing 
performance  models.} 

Models  human  behavior  as  interactions  among 
independent  computational  agents  representing 
interacting  individuals  or  cognitive  processes 
within  individuals.  Allows  both  sequentially 
dependent  and  parallel  task  performance  with 
order  determined  by  activation  levels  of  tasks  - 
without  an  explicit  executive  process.  Allows 
facile  interface  with  other  models. 

(Deutsch, 
MacMillan,  & 
Cramer,  1993) 
(Deutsch,  1998) 
(Cramer,  1998) 

PSI  (Not  an  acronym) 

Attempts  to  integrate  motivation  with  cognitive 
processes.  Based  on  three  levels  of  needs  that 
interact  to  determine  motive  strength  and  specific 
goal  behaviors:  System  needs  (water  and 
energy),  Preservation  level  (pain  avoidance), 
Information  level  (certainty,  competence, 
affiliation).  Action  strategies  first  seek 
automatized  skills,  then  knowledge -based 
behavior,  then  trial  and  error  to  satisfy  goals. 

(Bartl  &  Domer, 
1998) 

(Ritter,  et  al., 
2002) 

(http://www.uni- 

bamberg.de/~ba 

2dpl/psi.html) 

Situation  Awareness 
Model  for  Pilot-in-the- 
Loop  Evaluation 
(SAMPLE) 

Generalized  from  original  effort  to  model 
situation  awareness  of  pilots  and  air  crews  in  air 
combat.  Uses  cognitive  task  analyses,  pattern 
recognition  from  Klein’s  Recognition-Primed 
Decision-Making,  Endsley’s  three  levels  of 
awareness  (detection,  identification,  and 
prediction),  and  Rasmussen’s  three  tiers  of  action 
strategy  (skill-based  pattern  recognition, 
standardized  if-then  rules,  and  knowledge -based 
problem  solving)  to  provide  three  stages  of 
processing:  information  processing  (with  a 
continuous  state  estimator  and  a  discrete  event 
detector),  situation  assessment  (with  the 
information  fusion  and  reasoning  required  by 
multi-tasking),  and  decision-making  (with  a 
procedure  selector  and  a  procedure  executor). 
Output  includes  information  disparity,  situation 
awareness  disparity,  and  combat  advantage 
index. 

(Rasmussen, 

1983) 

(Endsley,  1988) 
(Klein,  1989) 
(Mulgund, 

Harper,  & 
Zacharias,  2002) 
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Model  Name 

Summary  Description 

Reference(s) 

State,  Operator,  And 
Result  (SOAR) 

Intended  as  a  comprehensive  model  of  human 
cognition  focused  on  operational  task  domains 
depicting  all  behavior  as  goal-driven  movement 
through  problem  spaces  that  define  states  and 
operators  for  the  task(s)  at  hand.  Uses  a  four¬ 
cycle  iterative  process  involving:  Input  (via 
human  perception),  Elaboration  (matches  if-then, 
condition-action  rules  in  long  term  memory  with 
those  in  working  memory  to  issue  proposals  for 
decision  making  and  direct  commands  for 
psychomotor  actions),  Output  (psychomotor 
execution),  Decision  (either  selects  operators  or 
identifies  ‘impasses’  requiring  a  new  subgoal 
until  all  impasses  are  resolved).  Uses  a  single 
process  for  long-term  memory,  learning,  task 
representation,  and  decision-making.  All 
learning  occurs  through  “chunking,”  which 
occurs  through  impasse  subgoaling  and 
resolution.  Emotions  arise  from  situation 
awareness  clarity  and  confusion.  Integrates 
individual  and  team  knowledge  and  allows  goals 
and  plans  to  be  shared  among  team  members. 

(Lewis,  2001) 

(http://ai.eecs.u 

mich.edu/soar) 

(http://www- 

2. cs. cmu.edu/afs 

/cs/project/ 

soar/public/ww 

w/home- 

page.html) 

(http://www.isLe 

du/soar/soar- 

homepage. 

html) 

(http://www.nott 

ingham.ac.uk/pu 

b/soar/ 

nottingham/soar 

-faq.html) 

(http://phoenix.il 

erts.ac.uk/~rmy/ 

cogarch.seminar 

/soar.html) 

How  might  these  models  contribute  to  the  development  of  computer-mediated  learning  and  performance 
aiding  environments?  As  suggested  above,  a  model  intended  to  support  education  and  training  needs  either 
an  implicit  or  explicit  model  of  cognition  if  it  is  to  assess  the  state  of  a  learner’s  knowledge,  skill,  and 
abilities.  To  do  this,  it  must  represent  memory  and  its  interactions  with  other  cognitive  functions  such  as 
perception  and  attention.  It  may  also  represent  such  cognitive  functions  as  decision-making  and  problem 
solving  as  well  as  cognitive  responses  to  the  environment  such  as  social  behavior  and  situation  awareness 
and/or  the  extent  of  cognitive  workload. 

However,  if  a  model  is  to  support  education  and  training,  it  is  not  enough  for  it  just  to  represent  the  current 
state  of  cognitive  processing.  It  must  also  represent  and  project  its  evolution  and  development.  In  short,  it 
must  include  a  model  of  human  learning.  Table  2,  taken  directly  from  Morrison,  summarizes  the  cognitive 
functions  covered  by  the  models  summarized  in  Table  1.  It  indicates  which  models  explicitly  represent  one 
or  more  of  the  following  cognitive  processes:  perception,  psycho  motor  performance,  attention,  situation 
awareness,  short-term  memory,  long-term  memory,  learning,  decision-making,  problem  solving,  cognitive 
workload,  emotional  behavior,  and  social  behavior. 

The  table  indicates  that: 

•  All  19  models  represent  decision  making  -  but  it  is  largely  the  reactive  form  of  decision  making  that  is 
captured  in  if-then  rules. 

•  All  1 9  models  represent  either  short-  or  long-term  memory. 

•  Perception  and  attention  were  well  represented  in  1 6  of  the  reviewed  models. 
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•  Although  only  4  of  the  models  explicitly  represented  situation  awareness,  the  functions  of  situation 
awareness  were  present  in  those  representing  perception  and  attention. 

•  Social  behavior  was  represented  in  only  5  of  the  models. 

•  Emotional  behavior  was  represented  in  only  3  of  the  models. 

•  Learning  was  represented  in  only  5  of  the  models  as  was  problem  solving.  Morrison  suggests  that  this 
limited  representation  may  be  due  to  the  nature  of  condition-response  production  models,  which  can 
react  to  the  situations  contained  in  anticipated  if-states,  but  which  may  not  adapt  well,  if  at  all,  to  the 
unanticipated  states  and  conditions  that  must  be  accommodated  in  learning  and  problem  solving. 

Table  2.  Cognitive  and  Behavioral  Functions  Represented  in  Models  Reviewed  by  Morrison  (2003) 


Acronym/ 

Abbreviation 

Cognitive  Function  Represented 

Perception 

Psychomotor 

Performance 

Attention 

Situation 

Awareness 

Working 

Memory 

Long-term 

Memory 

Learning 

Decision 

Making 

Problem 

Solving 

Cognitive 

Workload 

Emotional 

Behavior 

Social 

Behavior 

ACT 

X 

X 

X 

X 

X 

X 

X 

X 

ART 

X 

X 

X 

X 

X 

X 

APEX 

X 

X 

X 

X 

Brahms 

X 

X 

X 

X 

X 

CogAff 

X 

X 

X 

X 

X 

X 

COGNET 

X 

X 

X 

X 

X 

X 

X 

X 

CCT 

X 

X 

X 

X 

X 

COGENT 

X 

X 

X 

X 

CAPS 

X 

X 

X 

X 

X 

C-l  Theory 

X 

X 

X 

X 

DCOG 

X 

X 

X 

X 

X 

X 

EPIC 

X 

X 

X 

X 

X 

HOS 

X 

X 

X 

X 

X 

MIDAS 

X 

X 

X 

X 

X 

X 

X 

X 

Micro  Saint 

X 

X 

X 

X 

X 

OMAR 

X 

X 

X 

X 

X 

Psi 

X 

X 

X 

X 

X 

X 

X 

X 

X 

SAMPLE 

X 

X 

X 

X 

X 

X 

Soar 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Note:  An  “X”  entry  indicates  that  the  function  is  represented  by  the  model. 


The  five  models  judged  to  represent  learning  are:  ACT,  COGENT,  CAPS,  PS1,  and  Soar.  All  five  of  these 
models  also  represent  long-term  memory,  working  memory,  and  decision-making.  All  except  COGENT  also 
represent  perception,  psychomotor  performance,  and  attention. 


A  model  of  cognition  that  includes  learning  is  necessary  for  education  and  training  applications,  but  it  is  not 
sufficient.  A  model  of  learning  is  not  a  model  of  instruction.  All  19  models,  as  good  as  many  of  them  are, 
lack  this  component.  This  component  is  needed  to  suggest  links  between  specific  instructional  interventions 
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and  specific  learning  outcomes  -  teaching  strategies  that  reliably  bring  about  transitions  from  the  learner’s 
current  cognitive  state  to  one  capable  of  producing  the  intended  instructional  outcomes. 

Instructional  Systems  Development 

Attaining  a  “model  of  instruction”  centered  around  models  of  human  cognition  would  lead  to  what  might  be 
called  “engineering  of  instruction”  —instruction  viewed  as  neither  art  nor  science,  but  as  a  way  to  reliably 
and  efficiently  produce  specified  instructional  outcomes.  Such  a  capability  for  development  of  instructional 
and  performance  aiding  systems  should  be  based  on  empirically  derived  principles  that  can  be  realistically 
applied.  Outcomes  might  consist  of  general  objectives  such  as  ability  to  transfer  knowledge,  long-term 
retention  of  knowledge  and  skill,  motivation  to  continue  learning,  speed  of  response,  accuracy  of  response, 
and  so  forth.  The  outcomes  might  be  associated  with  more  specific  training  objectives  such  as  the  ability  to 
locate  single  component  failures  in  the  XYZ  power  supply,  pack  a  reserve  parachute,  or  devise  tactical  plans. 

Fragments  of  such  a  capability  for  engineering  instruction  have  been  identified  in  research  literature,  data, 
and  findings.  Work  is  needed  to  organize,  substantially  expand,  and  include  them  as  principles  to  be 
incorporated  in  our  current  models  of  cognition.  In  addition,  engineering  of  instruction  requires,  as  an 
essential  foundational  element,  robust  human  cognitive  models  in  order  for  the  training,  education  or 
performance  aiding  system  to  “know”  the  user  and  to  dynamically  adapt  to  the  user’s  state. 

What  Research  and  Development  Do  We  Need? 

This  brief  review  of  cognitive  models  applied  to  automated  instructional  and  performance  aiding  systems 
suggests  that  much  good  progress  has  been  made  but  that  much  remains  to  be  done.  We  do  not  yet  have  the 
models  we  need  to  fully  support  the  broad  range  of  human  behavior  required  for  simulations  we  now  use  in 
training,  analysis,  and  acquisition.  More  generally,  we  still  lack  the  comprehensive  models  we  need  to 
represent  subject  matter  expertise,  levels  of  student  learning,  and  most  especially  the  links  between  specific 
instructional  interventions  and  the  development  of  specifically  targeted  cognitive  abilities  needed  for 
competent  performance.  What  research  and  development  shoidd  we  pursue  to  achieve  short-  mid-  and  long¬ 
term  enhancements  in  the  state  of  the  art? 

This  issue  was  addressed  in  a  workshop  held  in  November  1999  to  assess  research  and  development  needed 
to  support  the  Department  of  Defense  Advanced  Distributed  Learning  initiative  (Final  Report,  1999),  in  a 
series  of  workshops  sponsored  in  2002-2003  by  the  Learning  Federation  (Learning  Federation,  2003),  and  in 
another  HFM  Symposium(Foster  and  Fletcher,  2002).  All  three  sources  cover  a  wide  range  of  issues  and 
organize  their  results  in  different  categories,  but  some  common  findings,  specifically  concerned  with 
research  necessary  for  the  development  of  cognitive  models,  emerge  from  them.  These  findings,  concerning 
cognitive  modeling,  are  summarized  in  Table  3  as  issues  along  with  some  specific  research  needed  to  meet 
these  goals  and  fill  gaps  in  our  current  capabilities. 
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Table  3.  Issues  and  Research  Requirements  for  the  Development  of  Cognitive  Modeling  Summarized  from 
Assessments  of  Learning  Technology  Needs 


Issue 

Research  Requirements 

Cognitive  Theory 

•  Representation  of  ‘higher  order’  cognitive  capabilities  (e.g.,  decision¬ 
making,  problem-solving,  meta-cognition,  pattern  recognition,  critical 
thinking,  situational  awareness,  teamwork). 

•  New  concepts  and  theories  of  cognition  and  cognitive  workload  based  on 
new  measurement  capabilities. 

•  Valid  and  verified  representation  of  expertise  and  its  development  in 
complex,  ill-structured  environments. 

•  Knowledge  representations  and  ontologies  that  allow  interoperability  and 
logical  operations  within  and  across  disciplines. 

Human  Behavior 
Representation 

•  Comprehensive  and  accurate  representation  of  individual  and  crew,  team, 
and  unit  expertise,  capabilities,  and  performance. 

•  Free,  cognitively  transparent  exchange  of  virtual  (avatar)  and  actual  users 
in  crew,  group,  team  learning 

Cognitive  Model 
Authoring 

•  Automated  development,  verification,  and  validation  of  cognitive 
models. 

•  Automated  processes  for  performing  cognitive  analysis  and  cognitive 
readiness  assessment. 

•  Automated  capture  of  expertise  —  self-generating,  self-modifying  data 
bases  built  from  cases  and  examples  of  successful  problem  solving  and 
decision-making. 

•  Principles  for  developing  physically  and  cognitively  realistic  avatars. 

User  Assessment 
and 

Representation 

•  New  forms  of  computer-administered  assessment  items  using  the  full 
display,  timing,  and  natural  language  understanding  capabilities  of 
technology. 

•  Generation  of  valid,  unobtrusive  near  real  time  assessment  from 
interactions  of  individuals,  teams,  crews,  and  units  with  the  learning  or 
performance  aiding  environment. 

•  Representation  of  subject  matter  misunderstandings  and  their  sources. 

•  Generation  and  use  of  questions  to  build  cognitive  profiles  of  users. 

•  Assessment  of  cognitive  workload. 
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Issue 

Research  Requirements 

•  Assessment  of  high-level  cognitive  skills  needed  to  deal  with 
unanticipated  and  unexpected  situations. 

Management  of 
Progress 

•  Ability  to  match  instructional  or  problem  solving  goals  with  current  state 
of  the  user  and  generate  or  select  optimal  tutorial  and/or  problem  solving 
strategies. 

•  Automated  principles  of  design  and  presentation  needed  to  ensure 
reliable  achievement  of  targeted  cognitive  state(s)  by  individuals,  crews, 
teams,  and  units. 

•  Automated  principles  for  the  development  of  higher-level  cognitive  skills 
such  as  creativity,  adaptability,  problem  solving,  and  situation  awareness. 

•  Comprehensive  understanding  of  meta-cognition  and  its  development. 

•  Comprehensive  understanding  of  incentive  management  and  its 
interaction  with  cognitive  development. 

•  Technology-based  tools  allowing  distributed  users  to  manage  their  own 
progress  and  problem  solving. 

•  Predictions  of  learning  rate  and  success  from  user  profile  information. 

User  Interface 

•  Management  of  user  dialogue  based  on  model  of  user  cognitive  abilities, 
style,  and  progress  toward  objective(s) 

The  efforts  suggested  by  Table  3  are  realistic  in  that  that  they  are  amenable  to  research  that  can  be  performed 
with  approaches  available  from  our  current  state  of  knowledge.  They  suggest  goals  that  can  be  achieved  to 
an  appreciable  degree  in  the  next  3-5  years.  Doing  so  will  be  worth  the  effort  and  will  return  much  more  to 
the  success  of  our  operational  capabilities  than  it  will  cost. 

The  value  of  cognitive  models  has  another,  increasingly  important  dimension.  The  current  world 
environment  presents  significant  challenges  to  our  capabilities  for  preparing  military  personnel  to  meet  them, 
and  thereby  to  our  capabilities  for  providing  military  education  and  training.  We  have  responded  in  ways 
that  have  proven  successful  in  the  past,  with  task  lists,  essential  task  lists,  mission  essential  task  lists,  and 
even  joint  mission  essential  task  lists.  These  task  lists  suggest  education  and  training  objectives  that  we 
know  how  to  meet. 

However,  the  current  asymmetric,  unpredictable  operational  environment  now  facing  our  military  personnel 
will  inevitably  present  situations  that  are  unexpected  and  for  which  they  may  be  little  prepared.  Our  people 
and  their  allies  will  have  to  respond  to  these  situations  with  agility,  flexibility,  creativity,  and  skillful 
leadership.  Their  readiness  to  acquire  the  additional  capabilities  needed  to  meet  these  unexpected, 
unforeseen  challenges  will  contribute  substantially  to  the  success  of  their  operations.  How,  then,  can  we  best 
prepare  our  people  to  expect  the  unexpected  and  deal  with  it  successfully?  Such  an  aspect  of  readiness  is  a 
cognitive  capability.  It  places  special  demands  on  our  ability  to  model  cognition  and  to  train  both  individuals 
and  units.  It  is  an  essential  component  of  what  we  have  called  cognitive  readiness  (Etter,  Foster,  &  Steele, 
2000),  and  a  combination  of  technology-based  education,  training,  and  performance  aiding  is  expected  to 
help  our  forces  achieve  it. 
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The  components  of  cognitive  readiness  cover  issues  that  include  the  following: 

Situation  awareness,  which  is  generally  defined  as  the  ability  to  perceive  oneself  in  relation  to  the  enemy  and 
the  environment.  Situation  awareness  has  been  shown  to  improve  with  practice  and  instructional  feedback. 

Memory,  which  is  described  as  an  active,  reconstructive  process  supported  by  two  underlying  theoretical 
mechanisms:  encoding  specificity,  which  stresses  the  importance  of  external  and  internal  cues,  and  transfer- 
appropriate  processing,  which  stresses  actions  performed  during  encoding  and  retrieval.  Trade-offs  exist 
between  instruction  used  to  enhance  retention  and  speed  of  initial  acquisition.  Conditions  of  learning, 
particularly  those  providing  overlearning,  can  be  designed  to  enhance  retention. 

Transfer  of  training,  which  is  described  as  the  ability  to  apply  what  is  learned  in  one  performance  context  to 
another.  Massive  amounts  of  practice  with  feedback  will  enhance  “low-road”  transfer  requiring  little 
cognitive  mediation.  Training  in  forming  mindful,  conscious  abstraction  will  enhance  “high  road”  transfer, 
which  requires  cognitive  mediation. 

Metacognition,  which  refers  to  the  executive  functions  of  thought,  particularly  those  pertaining  to  knowledge 
and  regulation  of  one’s  cognitive  processes  and  progress  toward  accepted  goals.  Metacognitive  skills  can  be 
enhanced  by  exercises  designed  to  increase  awareness  of  self-regulatory  processes. 

Automaticity.  which  refers  to  processes  that  are  performed  rapidly,  requiring  few  attentional  resources. 
Practice  with  feedback  and  overlearning  can  produce  automatic  processing  in  many  tasks. 

Problem  Solving,  which  transforms  goals  and  subgoals  into  a  plan  of  action  by  processes  such  as  trial-and- 
error,  proximity,  fractionation,  and  knowledge-based  referrals.  Techniques  for  problem  solving  matched  to 
goal  and  situation  categories  can  be  successfully  taught,  as  can  the  information  base  needed  for  “strong” 
problem  solving  methods,  which  depend  on  acquired  knowledge. 

Decision-Making,  which  is  described  as  the  selection  of  tactical  and  strategic  plans,  which  are  frequently 
primed  by  the  recognition  of  learned  patterns.  Formal  instruction  in  decision-making  techniques  may 
improve  the  quality  of  decisions,  but  some  aspects  of  successful  decision-making  are  determined  by 
individual  dispositions. 

Mental  Flexibility  and  Creativity,  which  may  be  cast  as  problem-solving,  applying  “strong”  methods  based 
on  acquired  knowledge  and  skills,  and  “weak”  methods,  used  for  poorly  defined,  ill-structured,  chaotic  tasks. 
Creativity  may  be  more  closely  associated  with  the  latter  “weak”  methods.  The  research  is  unclear  whether 
these  weak  methods  can  be  trained  directly.  It  seems  more  likely  that  the  facility  with  which  people  apply 
appropriate  weak  methods  (i.e.,  achieve  “creative  solutions”)  to  novel  situations  is  determined  by  native 
abilities. 

Leadership,  appears  to  consist  of  motivational  patterns  and  a  combination  of  technical,  conceptual,  and 
interpersonal  skills,  the  last  being  the  most  difficult  to  acquire  and  measure.  However,  technical  and 
conceptual  skills  that  are  needed  by  leaders  can,  to  an  appreciable  extent,  be  taught.  Interpersonal  skills  and 
patterns  of  motivation  required  for  leadership  appear  to  be  more  dependent  on  native  abilities  and  are  thus 
more  difficult  to  teach. 

Emotion,  must  be  channeled  and  controlled  if  military  personnel  are  to  perform  complex  tasks  under  the 
stress  and  confusion  that  accompany  modem  military  operations.  Deeply  engaging,  sensory  immersing 
simulations  provide  promise  for  training  warfighters  to  retain  critical  pieces  of  information  and  to  perform 
under  highly  stressful  conditions. 
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These  issues  have  been  discussed  extensively  in  research  literature  and  their  specific  relevance  to  cognitive 
readiness  have  been  discussed  by  Morrison  and  Fletcher  (2002).  The  points  to  be  emphasized  here  are  that 
(a)  assessment  and  development  of  the  capabilities  suggested  by  these  issues  will  key  on  the  adequacy  of  the 
cognitive  models  on  which  our  education,  training,  and  performance  aiding  are  based  and  (b)  the  adequacy  of 
our  cognitive  modeling  is  a  matter  of  first  importance  in  the  current  unpredictable  operational  environment. 

The  modeling  efforts  reviewed  in  this  paper  along  with  similar  efforts  involving  human  cognition  represent 
significant  opportunities  for  cooperative  research  by  the  NATO  community  concerned  with  the  human 
competence  that  is  an  essential  component  of  every  military  operation.  We  may  wish  to  rise  to  the 
opportunities  they  present. 
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